Retinal Mapping
Client - Nidek, CO, Ltd.
Images of the retina are used to diagnose and monitor the progress of a variety of diseases, including such leading causes of blindness as diabetic retinopathy, age-related macular degeneration, and glaucoma. Retinal image registration has a variety of applications. Registering a set of images taken during a single session with a patient can be used to form a single composite mosaic view of the entire retina. Multimodal registration can reveal the relationship between events seen on the surface of the retina and the blood flow shown in the angiography. Registering images taken weeks, months or years apart can be used to reveal changes in the retina at the level of small regions and individual blood vessels.
Retinal image registration is challenging. The images are projections of a curved surface taken from a wide range of viewpoints using an uncalibrated camera. The retina surface itself is curved significantly and any transformation process that aligns images of the retina must take that into account. The nonvascular surface of the retina is homogenous in healthy retinas and exhibits a variety of pathologies in unhealthy retinas. The pathologies are a poor choice for longitudinal registration because they may appear and disappear over time. Only the vasculature covers the entire retina and is relatively stable over time.
Procsssing of the single field of the retinal through a fundus camera produces the following result. |
|
Processing multiple fields of view of the retina for automatic panorama reconstruction results in the following. |
|
|
|
| The algorithms are robust and will work on strangely overlapping fields of view. |
|
| Once reconstructed, the entire retinal map can be analyzed further in terms of their blood vessels. |
|
|
Solution
The solution to the retinal image registration problem is an approach driven by the vascular structure. This can include the vessels themselves and their branching and crossover points. Differences in images, especially over longitudinal studies, can interfere with automatic registration. Images may appear dark or brighter between sessions. Some blood vessels may appear dark, while other vessels nearby may appear bright. Inclusions, pathologies and other artifacts can lead to false positives. Distortions in the image capturing process can lead to registration problems. Ultimately, the best process needs to have 1) very good initialization of seed points, 2) robust minimization algorithms that reduce the image to discrete data for the registration process, and 3) elimination or minimization of local errors, such as pathologies, geometric distortions and other local imaging issues.
In retinal imaging, the stable and prominent vascular structure should drive the minimization. Feature based techniques can be used in the minimization method where the images are aligned based on correspondences between automatically detected features. Feature based methods usually minimize an objective function of the distances between vascular landmarks correspondences. To narrow down the search space to the correct domain of convergence, matched landmarks, such as the optic disk and the fovea, can be used in the initialization. Alternatively, manually indicated landmarks and other registration points inherent in the vasculature can be used on overlapped regions of images. In all landmark-based algorithms, the major problem is finding and matching a set of consistent landmarks to sufficiently constrain the transformation needed for accurate registration. This is particularly a problem for nonlinear transformations, low image overlap, poor quality images, and longitudinal changes.
Delineation of the vasculature, also known as tracing, uses algorithms to follow the vessels, along its centerline. Every centerline point is represented by location, tangent direction and width to the sides of the transected blood vessel. While most fundus images in the retinal image alignment project are basically flattened 2D images, this description can be extended into a 3D description by indication a 3D tangent direction and a location in 3 space. After the entire tree has been analyzed, a series of data structures is rendered that represent the tree via the node points or branching, and the characteristics of the branch points. Characteristic landmarks of the vasculature are described by each branch radiating out from an origin, bifurcation or branch points and lengths in between branch points. Landmarks are invariant to rotation, translation and scaling of the image and serve as excellent correspondence points in the overlap regions of the images for the registration phase.
The overall procedure to align images is:
- Trace the vasculature of each image
- Characterize local landmarks of the vasculature
- Pair corresponding landmarks
- Perform transformations between image pairs